Consonance and the Closure Method in Multiple Testing

Consider the problem of testing s null hypotheses simultaneously. In order to deal with the multiplicity problem, the classical approach is to restrict attention to multiple testing procedures that control the familywise error rate (FWE). The closure method of Marcus et al. (1976) reduces the problem of constructing such procedures to one of constructing single tests that control the usual probability of a Type 1 error. It was shown by Sonnemann (1982, 2008) that any coherent multiple testing procedure can be constructed using the closure method. Moreover, it was shown by Sonnemann and Finner (1988) that any incoherent multiple testing procedure can be replaced by a coherent multiple testing procedure which is at least as good. In this paper, we first show an analogous result for dissonant and consonant multiple testing procedures. We show further that, in many cases, the improvement of the consonant multiple testing procedure over the dissonant multiple testing procedure may in fact be strict in the sense that it has strictly greater probability of detecting a false null hypothesis while still maintaining control of the FWE. Finally, we show how consonance can be used in the construction of some optimal maximin multiple testing procedures. This last result is especially of interest because there are very few results on optimality in the multiple testing literature. DOI: https://doi.org/10.2202/1557-4679.1300 Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-47262 Accepted Version Originally published at: Romano, Joseph P; Shaikh, Azeem M; Wolf, Michael (2011). Consonance and the closure method in multiple testing. The International Journal of Biostatistics, 7(1):online. DOI: https://doi.org/10.2202/1557-4679.1300 Institute for Empirical Research in Economics University of Zurich Working Paper Series ISSN 1424-0459 Working Paper No. 446 Consonance and the Closure Method in Multiple Testing Joseph P. Romano, Azeem M. Shaikh and Michael Wolf September 2009 Consonance and the Closure Method in Multiple Testing Joseph P. Romano Departments of Statistics and Economics Stanford University [email protected] Azeem M. Shaikh Department of Economics University of Chicago [email protected] Michael Wolf Institute for Empirical Research in Economics University of Zurich [email protected]